The present invention relates to the field of predictive video coding, particularly to ensuring a proper match between encoder and decoder states.
Most current video compression algorithms require keeping state information between the decoding of pictures. One obvious example is the reference picture(s) used for inter picture prediction. When errors have occurred during the bit stream transmission from the encoder to the decoder this decoder state is normally corrupted. However, only in a few cases a decoder can determine the presence of corruption from the bit stream itself. In other cases, external means (e.g. sequence numbering of transmission packets) can also be used to determine presence of possible corruption.
Corruption of the decoder state can occur due to erroneous transmission of the bit stream. At the decoder, while most transport stacks contain indication of possible bit stream corruption, it is unclear in which data structure the corruption occurs (if at all possible to determine that corruption occurred). For example, damaged slice data can lead to a corrupted reference picture used for future prediction, and a damaged or lost in-band parameter set transmission can lead to a lost or damaged parameter set, that may be referenced later.
Before this invention, no mechanism was known which allowed an encoder to inform a decoder about its internal state, in a lightweight manner. Also, previously no mechanism was known that allowed a decoder to inform an encoder about its internal state, without inferring a corruption (or non-corruption). In other words, no means existed by which a decoder can (for example periodically) inform an encoder about its state without having performed error detection and explicitly sending information that positively or negatively informs the encoder about corruption.
The following decoder to encoder feedback messages are known from prior art. They all infer corruption or non-corruption implicitly. In order to generate these prior art messages; the error detection has to be performed by the message sender (the decoder).
NEWPRED reference picture feedback messages (see for example U.S. Pat. No. 6,621,868 and Kimata, H., et al, “Study on Adaptive Reference Picture Selection Coding Scheme for the NEWPRED Receiver-Oriented Mobile Visual Communication,” IEEE Global Telecommunications Conference, Nov. 8-12, 1998. (8 pages), ITU-T Rec. H.263 Annex N): The decoder can inform an encoder about the known corruption or known non-corruption of a reference picture with a certain ID. The encoder can use this information to use an older reference picture for inter picture prediction, which is known to be uncorrupted (at the decoder). However, the decoder cannot simply send information about its state and let the encoder decide whether corruption or non-corruption exists. In other words, the burden of error detection lies at the decoder, not at the encoder.
Packet loss back channel signaling (e.g. ARQ): These techniques signal back the non-arrival of a packet in order to trigger re-sending. This is information wherein the state of a receiver (“packets x, y, z are lost”) is conveyed. However, this is not a “state” in the sense of a video decoder state that pertains over data entities other than packets.
Full Intra Request, picture loss indication, slice loss indication, and similar video-related mechanisms indicating corruption: These are available in many different standards, e.g. in the “Extended RTP Profile for RTCP-based Feedback (RTP/AVPF)” Internet Draft (AVPF, please see http://http://www.ietf.org/internet-drafts/draft-ietf-avt-rtcp-feedback-11.txt), or in the ITU-T Rec. H.245. All these require the error detection to be performed at the decoder.
It is the object of the present invention to provide means for validating a proper match between encoder and decoder states. In the present invention error detection is enabled to be handled by the message receiver, in contrast to the above identified prior art requiring the error detection to be performed at the message sender. Furthermore state of the art feedback messages infer corruption or non-corruption, whereas messages according to the invention are “neutral”.